Action for building docker images automatically by j143 · Pull Request #1441 · apache/systemds

j143 · 2021-11-06T14:03:34Z

Tasks

1. Run nightly build as cron job
2. run test build for each commit
3. version build for release version

How to Review?

Published artifact at https://hub.docker.com/r/return01/test-ghactions-systemds

Usage test:

janardhan@cloudshell:~ (ai-experiments-001)$ docker run -it --rm return01/test-ghactions-systemds
Hello from SystemDS
SystemDS Statistics:
Total execution time:           0.020 sec.

Secrets: DOCKERHUB_USER and DOCKERHUB_TOKEN
These two variables are already set by INFRA. No need to set from our end.

https://issues.apache.org/jira/browse/INFRA-22430

Baunsgaard · 2021-11-06T15:26:56Z

I think we can split the images a bit and reduce builds.

Build max once a day.
Only build the test image if we change R dependencies or the pom.xml
Only build the other images if there is changes in src.

Thanks for the help!

Baunsgaard · 2021-11-26T13:49:42Z

+    - name: Checkout
+      uses: actions/checkout@v2
+
+    - name: Set up QEMU


QEMU is for ?

It is an add-on, incase we want to build multi arch images. Optional though!

https://github.com/docker/buildx/#building-multi-platform-images

usage

$ docker buildx build --platform linux/amd64,linux/arm64 .

I do not have a about whether we need this. Perhaps, federated feature
might need this (I do not have good understanding yet).

hmm, then for now lets not include it if it is not needed

Baunsgaard · 2021-11-26T13:50:33Z

 	rm -r target/maven-archiver && \
 	rm -r target/systemds-** && \
-	rm -r docker && \
+	# rm -r docker && \


you cannot just comment out a line in this multi-line instruction, then everything after it does not execute.

Suggested change

# rm -r docker && \

rm -r docker && \

Not sure what you mean?
just for information the reason why i have this line is to remove the docker folder from the internal copied systemds repository, it does not make a difference for the building of the docker image except it makes the end result slightly smaller.

so therefore, don't out comment it :)

Baunsgaard · 2021-11-26T13:51:23Z

-COPY docker/mountFolder/main.dml /input/main.dml
+RUN mkdir /input && echo 'print("Hello from SystemDS")' > /input/main.dml
+
+# COPY docker/mountFolder/main.dml /input/main.dml


i'm all for running it but don't remove the copy, so that the only change you make it to add the RUN line.

I am having error with running COPY

this is happening because of your out commented line mentioned above.

this one:

# rm -r docker && \

Baunsgaard · 2021-12-13T17:10:47Z

I think it is fine if we just add the on commit part of the docker for now,
If it is still working, we should start merging it in.

Baunsgaard · 2022-01-08T09:36:26Z

@j143
Is the only thing missing not just a Infra ticket to add keys to push the images?

j143 · 2022-01-08T13:25:36Z

Is the only thing missing not just a Infra ticket to add keys to push the images?

There are already secrets available.

the problem seems to be COPY docker/mountFolder/main.dml /input/main.dml statement in the Dockerfile. Let me give a couple of hours to it. :)

#9 [4/4] COPY docker/mountFolder/main.dml /input/main.dml
#9 ERROR: failed to walk /tmp/buildkit-mount645512887/docker/mountFolder: lstat /tmp/buildkit-mount645512887/docker/mountFolder: no such file or directory

j143 added 4 commits November 6, 2021 00:32

Create docker-pub.yml

f944d6e

correct syntax

10c2707

add run commands

ab905d6

add only testsysds docker image

4e3924f

add publish action

532568b

j143 requested a review from Baunsgaard November 24, 2021 16:19

j143 changed the title ~~[WIP] Action for building docker images automatically~~ Action for building docker images automatically Nov 24, 2021

Baunsgaard reviewed Nov 26, 2021

View reviewed changes

j143 added 2 commits January 8, 2022 22:14

fix issues with file context

0cc9187

Rename docker-pub.yml to docker-cd.yml

e3dd674

j143 merged commit de8a342 into apache:main Jan 8, 2022

j143 deleted the docker-action-pub branch January 8, 2022 16:53

Uh oh!

Conversation

j143 commented Nov 6, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Baunsgaard commented Nov 6, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

j143 Nov 26, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

j143 Nov 26, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Baunsgaard commented Dec 13, 2021

Uh oh!

Baunsgaard commented Jan 8, 2022

Uh oh!

j143 commented Jan 8, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

j143 commented Nov 6, 2021 •

edited

Loading

j143 Nov 26, 2021 •

edited

Loading

j143 Nov 26, 2021 •

edited

Loading

j143 commented Jan 8, 2022 •

edited

Loading